How Journaling Works

History > VHS Journal Files > How Journaling Works

How Journaling Works

If daily journaling is enabled, the VHS will create a live journal file in the service data folder. The VHS will actively write database changes to this file as they occur throughout the day. These changes are compressed before writing to disk to reduce the overall journal file size. When the day ends (i.e. 12:00 a.m. midnight in local server time), the VHS will close the existing live journal file, mark it as ready to be sealed, and moves the file to the configured journal files directory for storage. Then a new live journal file will be created in the service data folder and the service will begin writing new changes to that file.

A maintenance task is then queued up in the VHS to seal and close the previous day’s file that was moved into the journal directory at midnight. This process happen a short time after moving the pended file (within a minute after opening a new live journal file for the day). The seal process allows the VHS to write some additional summary information to the journal file and mark that day’s changes as complete. The VHS will never again write data to a sealed file, so the integrity of that day’s changes are preserved. Once a journal is sealed, it can be backed up using traditional IT backup strategies and also used for data recovery using the VHS Recovery Utility.

Note: A daily journal file contains all changes to the VHS database during the current day. The timestamp of the historical values being added or changed to the database may not necessarily match the date of the journal file. For example, values added to the VHS today with a timestamp of four months ago will be written as changes to today’s journal file, because that is the day on which the addition to the VHS occurred. So if attempting to recover these value additions from journal files, you would find those values adds within today’s journal rather than a journal file from four months ago.

Journal files are named in the following format: Journal_[DDDD]SITE.SERVICE_YYYYMMDDHHMMSS. The [DDDD]SITE.SERVICE, represents the Domain ID and Site and Service name of the VHS that created the journal file. The time suffix indicates the local time that the journal file was created. A live journal files that the VHS is currently writing to will have a file extension of .LVJ. Journal files that are pending a seal operation after a new day begins will have a file extension of .JPS, and sealed journal files have a .JNL file extension.

Journal files are kept in the journal directory for up to a maximum of 35 days depending on configuration settings. Once the journal directory contains 35 days of journal files, when a new journal file is created, the oldest journal file will be deleted. To keep journal files longer than the configured retention period, you must back the closed journal files up to another location outside of the journal directory. The retention period limits the number of days of journal files, not the number of files. For example, 10 days of journal may be contained in 20 journal files. Conditions that produce more than one journal file per day are discussed below in Multiple Journal Files for a Day.

Multiple Journal Files for a Day

The VHS will always create a new live journal file at midnight of each day. However, there are a few conditions that cause the VHS to close a live journal file and begin a new one at some other point in the day.

When one of these conditions occur, the result is multiple journal files for a single day:

If the VHS starts up after a bad shutdown, it will create a new journal file with the timestamp of the time the new file was created. The previous live journal file that was not properly closed will be moved to the journal directory and sealed. The header of the journal file will be marked to indicate a bad shutdown had occurred. While the VHS Quick Recovery utility will be able to read this file to recover the changes contained within, it is possible that the most recent changes at the end of the file were corrupted due to the bad shutdown. The previous changes in the file should be uncontaminated because changes in journal files are always appended to the end of the file, and blocks of changes within the file have a Cyclic Redundancy Check (CRC) written to ensure integrity of data.
An info item can be sent to the VHS to trigger it to close out the existing live journal file and start a new one for the day. The info item is VHS_START_JOURNAL, and can be sent with a Set Info Keyword Task in the MSS or through script via the Global Functions SetGenserveInfo function. These methods are used to schedule creation of journal files at a rate greater than once a day if desired. See the VHS SVCINFO Security Event for information about security access levels for performing on-demand VHS journal file seals.

Note: If using a Set Info Keyword Task, the MSS issuing the task must match the MSS service configured in the VHS configuration file.